SortMagic
by
Shawn M. Gordon
President
S.M.Gordon & Associates
Introduction
You might well ask why I am reviewing a sort utility, after all what else is it going to do except sort? In general this is true, however I recently came across a sort utility with a difference. It was also a twist of fate that made me even look at it in the first place. Perhaps you have seen ads for Magic Softwares MagicSort, I know I had, but the ad didn’t really do it justice. It wasn’t until I was talking with an acquaintance, and he mentioned that he knew the guy who wrote it. So one thing led to another and it sounded interesting enough to warrant looking at and then reviewing.
At it’s most basic SortMagic works as a replacement for SORT.PUB.SYS. It runs significantly faster on my Classic, but I am not sure how it compares to HP’s new sort on MPE/iX 4.0. It also supports SIP, which is Sort In Place. This is much more forgiving on disk space consumption because no scratch file get’s generated. If for some reason it fails your file is still there in a non-corrupt state, you just need to do the sort again. There are some other exciting things that are included in the software, so read on to find out.
Well I’ve said it before, but here it goes again. I tested the software on my series 70 HP 3000 with 12 Meg of main memory and 3 Gig of disk space that is 35% free. I used both Probe from SSI and the sort programs statistics to measure performance. Also, whenever I refer to IMAGE, this also goes for TurboImage, TurboImage/XL and TurboImage/iX.
Features
First things first; I did a few sorts to compare against SORT/3000, here are the results:
# of records 100131
CPU seconds Wall seconds
SORT/3000 206 1050
SortMagic 112 420
In terms of total time SortMagic was almost twice as fast. The above examples was working on a file with a record length of 256 bytes with two different keys. A two byte key and a twenty byte key.
As an example of SortMagics ability to do MR NOBUF I/O, I compared it against a QUERY serial read, a QUERY serial read that was going through IOMATE and then a selection using SortMagic, the results follow;
# of records 340000
Query Query/IOMate SortMagic
50 minutes 12 minutes 8 minutes
As you can see there is a pretty drastic difference, and this didn’t even take into account sorting the information afterward.
What’s great when you are going against any IMAGE files is that whenever it is prompting for Sets or Items you can type ? at any time and get a list of whatever is appropriate for the prompt. I asked if they would add an option to present data items in sorted order and they said they would probably add it. You get a nice full description of everything, names, types, lengths and such, so it makes it easy to find and select items.
A really great addition with the software is two callable routines. The first is a sorting routine that does what you expect it to do, sort. It works as well as the SORT.PUB.MAGIC program works and it gives you programmatic access. The second routine, in my opinion, is so cool that it’s almost worth buying the software just to get it. It is a callable routine that will do MR NOBUF reads against an IMAGE data set, use selection criteria based on the IMAGE item names and using all sorts of combinations of boolean operators, sort the file, and leave it in a file for you to access. I don’t know about you, but almost everything I write does a load up routine that reads an IMAGE data set and then sorts it. To be able to call just one routine and get super fast reads and sorts at the same time is just to hip. The status array returns the number of records written to the file, CPU and Wall milliseconds, number of passes made by the sort process and the record length of the output record.
I found that using DBSORT to retrieve and sort data set records was at least twice as fast as the methodology that I had been using. This consisted of serially reading a data set into a file and then closing the file and sorting it. This is a significant improvement, not to mention the difference in the size of the source code between the two methods.
My only complaint with these two routines is that the documentation should contain a couple of coding examples in various languages like FORTRAN and COBOL, this would reduce the amount of trial and error that could go on.
What is convenient is the way the parameters are passed to the DBSORT routine. They follow the same guidelines as IMAGE so you probably won’t have to create a bunch of new variables in your code.
Usability (also installation)
The software uses the standard, restore a job, stream the job, put the tape back online, that was made popular years ago, so installation is a snap. The program SORT.PUB.MAGIC is extremely easy to use, in general you will find that you will almost never look at the manual. The software takes less than 2k sectors of disk space with about 1k each for either the CM or NM versions, both are distributed on the tape.
Reliability
In the time that I used the software it never caused a system or user hang, and my output was always what I expected. The error handling in the program is good, but not great, there are times especially when doing selections where you could easily put the wrong syntax but the software doesn’t really tell you how it should be worded, even though it tell’s you it’s wrong.
Performance
The performance of the software is terrific. It can really tear through a sort as well as cranking out some very respectable MR NOBUF I/O when doing selecting. The impact on the CPU isn’t really any different than anything else that is doing sorting or file reads, as a matter of fact it seemed to be more forgiving that SORT/3000.
Supportability (including Doc)
The support is very good and the manual is mercifully small. The on-line help is so good you will probably only look at the doc a couple of times. However the explanation for defining complex sort keys or item selection is a little less intuitive. The documentation for the two callable routines DBSORT and MAGICSORT are good, but I would have liked to have seen a few examples in various languages, as well as an explanation of how you are supposed to link them to your program. I didn’t have a problem with it, but some people
Summary
Although the finest piece of this software is it’s ability to sort very quickly, I would say it would be better defined as a file slice and dice utility. I happen to find a slice and dice utility MUCH more interesting and useful than just sorting. The fact that the software has incredible speed when it comes to sorting and reading through files makes it just that much more useful when it comes to slicing and dicing files. I would have to say it is more along the lines of a SuperTool product than anything else. It doesn’t have all the features, but it has a better interface.
This package isn’t for everyone, however it is very useful for a variety of tasks. If you aren’t doing a lot of sorting you may want to look at it just for it’s ability to create input files for other programs. I especially like the two callable routines that allowed you to do different types of sorting.
There are just a few things I would like to see added to the software. One is a total graphic environment for tagging data items or bytes from a record layout and creating an output file format. Another would be a utility that would allow you to patch existing executable code to force it to use their SORT procedures the way SortMate from RunningMate can. The last would be frequency analysis. The only thing that I am aware of (doesn’t mean it doesn’t exist though) that does frequency analysis is the contributed program SELCOPY. Turns out that frequency analysis is a hobby of the software designer so don’t be surprised to see it turn up by the time this review hits the streets.
At-a-Glance box
SortMagic version 920115
Magic Software
1003 Main St.
Grand Junction, CO 81502
800-MAGIC55
800-MAGIC54 FAX
Call, write or FAX for more information or a demo.
Price is $1500 on Classics and ranges from $2500-$4500 on Spectrum.
Support is 10% of purchase price.
Good Points
VERY fast sorting and reading
Nice interface for IMAGE
Bad Points
Can’t modify executable code to call the routines
No Frequency analysis